text:
server returned HTTP status 500 Internal Server Error

Hello,
thank you for your work. Sadly prometheus reports a "server returned HTTP status 500 Internal Server Error" in target overview. If I request
http://10.100.165.16:9633/metrics
via Browser, I get:
An error has occurred while serving metrics:

10 error(s) occurred:
* collected metric "smartctl_device_smartctl_exit_status" { label:<name:"device" value:"" > label:<name:"model_family" value:"" > label:<name:"model_name" value:"" > label:<name:"serial_number" value:"" > gauge:<value:2 > } was collected before with the same name and label values
* collected metric "smartctl_device" { label:<name:"ata_additional_product_id" value:"unknown" > label:<name:"ata_version" value:"" > label:<name:"device" value:"" > label:<name:"firmware_version" value:"" > label:<name:"interface" value:"" > label:<name:"model_family" value:"" > label:<name:"model_name" value:"" > label:<name:"protocol" value:"" > label:<name:"sata_version" value:"" > label:<name:"serial_number" value:"" > gauge:<value:1 > } was collected before with the same name and label values
* collected metric "smartctl_device_capacity_blocks" { label:<name:"device" value:"" > label:<name:"model_family" value:"" > label:<name:"model_name" value:"" > label:<name:"serial_number" value:"" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_capacity_bytes" { label:<name:"device" value:"" > label:<name:"model_family" value:"" > label:<name:"model_name" value:"" > label:<name:"serial_number" value:"" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_block_size" { label:<name:"blocks_type" value:"logical" > label:<name:"device" value:"" > label:<name:"model_family" value:"" > label:<name:"model_name" value:"" > label:<name:"serial_number" value:"" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_block_size" { label:<name:"blocks_type" value:"physical" > label:<name:"device" value:"" > label:<name:"model_family" value:"" > label:<name:"model_name" value:"" > label:<name:"serial_number" value:"" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_interface_speed" { label:<name:"device" value:"" > label:<name:"model_family" value:"" > label:<name:"model_name" value:"" > label:<name:"serial_number" value:"" > label:<name:"speed_type" value:"max" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_interface_speed" { label:<name:"device" value:"" > label:<name:"model_family" value:"" > label:<name:"model_name" value:"" > label:<name:"serial_number" value:"" > label:<name:"speed_type" value:"current" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_power_on_seconds" { label:<name:"device" value:"" > label:<name:"model_family" value:"" > label:<name:"model_name" value:"" > label:<name:"serial_number" value:"" > counter:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_power_cycle_count" { label:<name:"device" value:"" > label:<name:"model_family" value:"" > label:<name:"model_name" value:"" > label:<name:"serial_number" value:"" > counter:<value:0 > } was collected before with the same name and label values

smartctl version is:
[root@NxOsLab004 ~]# smartctl --version
smartctl 7.0 2018-12-30 r4883 [x86_64-linux-3.10.0-957.1.3.el7.x86_64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

The smartctl-exporter reports a warning, because return value of smartctl is 32, but should be irrelevant.
[root@NxOsLab004 ~]# ./smartctl_exporter --config=./smartctl_exporter.yaml --debug
[Debug] Parsed options: {{10.100.165.16:9633 /metrics %!s(bool=false) /usr/sbin/smartctl 20s 20s [/dev/sda /dev/sdb /dev/sdc /dev/sdd]}}
[Debug] Collecting S.M.A.R.T. counters, device: /dev/sda
[Warning] S.M.A.R.T. output reading error: exit status 32
[Debug] Collecting S.M.A.R.T. counters, device: /dev/sdb
[Warning] S.M.A.R.T. output reading error: exit status 32
[Debug] Collecting S.M.A.R.T. counters, device: /dev/sdc
[Warning] S.M.A.R.T. output reading error: exit status 2
[Debug] Collecting S.M.A.R.T. counters, device: /dev/sdd
[Warning] S.M.A.R.T. output reading error: exit status 2
[Info] Starting on 10.100.165.16:9633/metrics

Its possible to avoid that behavior?
Regards.

Sorry for the long answer...
Can you send me result of smartctl --json --xall /dev/sda > /tmp/sda.json, json files for all devices?

I'm getting the same:
An error has occurred while serving metrics:

16 error(s) occurred:
* collected metric "smartctl_device_statistics" { label:<name:"device" value:"/dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3NY2V" > label:<name:"model_family" value:"Seagate Exos X14" > label:<name:"model_name" value:"ST12000NM0008-2H3101" > label:<name:"serial_number" value:"ZHZ3NY2V" > label:<name:"statistic_flags_long" value:"valid" > label:<name:"statistic_flags_short" value:"V---" > label:<name:"statistic_name" value:"Vendor Specific" > label:<name:"statistic_table" value:"Vendor Specific Statistics" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_statistics" { label:<name:"device" value:"/dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3NY2V" > label:<name:"model_family" value:"Seagate Exos X14" > label:<name:"model_name" value:"ST12000NM0008-2H3101" > label:<name:"serial_number" value:"ZHZ3NY2V" > label:<name:"statistic_flags_long" value:"valid" > label:<name:"statistic_flags_short" value:"V---" > label:<name:"statistic_name" value:"Vendor Specific" > label:<name:"statistic_table" value:"Vendor Specific Statistics" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_statistics" { label:<name:"device" value:"/dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3P3E3" > label:<name:"model_family" value:"Seagate Exos X14" > label:<name:"model_name" value:"ST12000NM0008-2H3101" > label:<name:"serial_number" value:"ZHZ3P3E3" > label:<name:"statistic_flags_long" value:"valid" > label:<name:"statistic_flags_short" value:"V---" > label:<name:"statistic_name" value:"Vendor Specific" > label:<name:"statistic_table" value:"Vendor Specific Statistics" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_statistics" { label:<name:"device" value:"/dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3P3E3" > label:<name:"model_family" value:"Seagate Exos X14" > label:<name:"model_name" value:"ST12000NM0008-2H3101" > label:<name:"serial_number" value:"ZHZ3P3E3" > label:<name:"statistic_flags_long" value:"valid" > label:<name:"statistic_flags_short" value:"V---" > label:<name:"statistic_name" value:"Vendor Specific" > label:<name:"statistic_table" value:"Vendor Specific Statistics" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_statistics" { label:<name:"device" value:"/dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3P9P1" > label:<name:"model_family" value:"Seagate Exos X14" > label:<name:"model_name" value:"ST12000NM0008-2H3101" > label:<name:"serial_number" value:"ZHZ3P9P1" > label:<name:"statistic_flags_long" value:"valid" > label:<name:"statistic_flags_short" value:"V---" > label:<name:"statistic_name" value:"Vendor Specific" > label:<name:"statistic_table" value:"Vendor Specific Statistics" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_statistics" { label:<name:"device" value:"/dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3P9P1" > label:<name:"model_family" value:"Seagate Exos X14" > label:<name:"model_name" value:"ST12000NM0008-2H3101" > label:<name:"serial_number" value:"ZHZ3P9P1" > label:<name:"statistic_flags_long" value:"valid" > label:<name:"statistic_flags_short" value:"V---" > label:<name:"statistic_name" value:"Vendor Specific" > label:<name:"statistic_table" value:"Vendor Specific Statistics" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_statistics" { label:<name:"device" value:"/dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3Q95P" > label:<name:"model_family" value:"Seagate Exos X14" > label:<name:"model_name" value:"ST12000NM0008-2H3101" > label:<name:"serial_number" value:"ZHZ3Q95P" > label:<name:"statistic_flags_long" value:"valid" > label:<name:"statistic_flags_short" value:"V---" > label:<name:"statistic_name" value:"Vendor Specific" > label:<name:"statistic_table" value:"Vendor Specific Statistics" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_statistics" { label:<name:"device" value:"/dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3Q95P" > label:<name:"model_family" value:"Seagate Exos X14" > label:<name:"model_name" value:"ST12000NM0008-2H3101" > label:<name:"serial_number" value:"ZHZ3Q95P" > label:<name:"statistic_flags_long" value:"valid" > label:<name:"statistic_flags_short" value:"V---" > label:<name:"statistic_name" value:"Vendor Specific" > label:<name:"statistic_table" value:"Vendor Specific Statistics" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_statistics" { label:<name:"device" value:"/dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3QGV5" > label:<name:"model_family" value:"Seagate Exos X14" > label:<name:"model_name" value:"ST12000NM0008-2H3101" > label:<name:"serial_number" value:"ZHZ3QGV5" > label:<name:"statistic_flags_long" value:"valid" > label:<name:"statistic_flags_short" value:"V---" > label:<name:"statistic_name" value:"Vendor Specific" > label:<name:"statistic_table" value:"Vendor Specific Statistics" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_statistics" { label:<name:"device" value:"/dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3QGV5" > label:<name:"model_family" value:"Seagate Exos X14" > label:<name:"model_name" value:"ST12000NM0008-2H3101" > label:<name:"serial_number" value:"ZHZ3QGV5" > label:<name:"statistic_flags_long" value:"valid" > label:<name:"statistic_flags_short" value:"V---" > label:<name:"statistic_name" value:"Vendor Specific" > label:<name:"statistic_table" value:"Vendor Specific Statistics" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_statistics" { label:<name:"device" value:"/dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3R74X" > label:<name:"model_family" value:"Seagate Exos X14" > label:<name:"model_name" value:"ST12000NM0008-2H3101" > label:<name:"serial_number" value:"ZHZ3R74X" > label:<name:"statistic_flags_long" value:"valid" > label:<name:"statistic_flags_short" value:"V---" > label:<name:"statistic_name" value:"Vendor Specific" > label:<name:"statistic_table" value:"Vendor Specific Statistics" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_statistics" { label:<name:"device" value:"/dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3R74X" > label:<name:"model_family" value:"Seagate Exos X14" > label:<name:"model_name" value:"ST12000NM0008-2H3101" > label:<name:"serial_number" value:"ZHZ3R74X" > label:<name:"statistic_flags_long" value:"valid" > label:<name:"statistic_flags_short" value:"V---" > label:<name:"statistic_name" value:"Vendor Specific" > label:<name:"statistic_table" value:"Vendor Specific Statistics" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_statistics" { label:<name:"device" value:"/dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3SZCM" > label:<name:"model_family" value:"Seagate Exos X14" > label:<name:"model_name" value:"ST12000NM0008-2H3101" > label:<name:"serial_number" value:"ZHZ3SZCM" > label:<name:"statistic_flags_long" value:"valid" > label:<name:"statistic_flags_short" value:"V---" > label:<name:"statistic_name" value:"Vendor Specific" > label:<name:"statistic_table" value:"Vendor Specific Statistics" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_statistics" { label:<name:"device" value:"/dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3SZCM" > label:<name:"model_family" value:"Seagate Exos X14" > label:<name:"model_name" value:"ST12000NM0008-2H3101" > label:<name:"serial_number" value:"ZHZ3SZCM" > label:<name:"statistic_flags_long" value:"valid" > label:<name:"statistic_flags_short" value:"V---" > label:<name:"statistic_name" value:"Vendor Specific" > label:<name:"statistic_table" value:"Vendor Specific Statistics" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_statistics" { label:<name:"device" value:"/dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3WHE7" > label:<name:"model_family" value:"Seagate Exos X14" > label:<name:"model_name" value:"ST12000NM0008-2H3101" > label:<name:"serial_number" value:"ZHZ3WHE7" > label:<name:"statistic_flags_long" value:"valid" > label:<name:"statistic_flags_short" value:"V---" > label:<name:"statistic_name" value:"Vendor Specific" > label:<name:"statistic_table" value:"Vendor Specific Statistics" > gauge:<value:0 > } was collected before with the same name and label values
* collected metric "smartctl_device_statistics" { label:<name:"device" value:"/dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3WHE7" > label:<name:"model_family" value:"Seagate Exos X14" > label:<name:"model_name" value:"ST12000NM0008-2H3101" > label:<name:"serial_number" value:"ZHZ3WHE7" > label:<name:"statistic_flags_long" value:"valid" > label:<name:"statistic_flags_short" value:"V---" > label:<name:"statistic_name" value:"Vendor Specific" > label:<name:"statistic_table" value:"Vendor Specific Statistics" > gauge:<value:0 > } was collected before with the same name and label values

Daemon output:
root@lstor1:/etc/service/smartctl_exporter# ./run
[Verbose] Read options from /etc/smartctl_exporter.yaml

[Debug] Parsed options: {{[::]:9666 /metrics %!s(bool=false) /usr/local/sbin/smartctl 20s 20s [/dev/disk/by-id/scsi-SATA_PNY_CS900_120GB_PNY0520228055010E219 /dev/disk/by-id/scsi-SATA_PNY_CS900_120GB_PNY0520228055010E21B /dev/disk/by-id/scsi-SATA_ST10000NM0016-1T_ZA2CZA35 /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3NY2V /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3P3E3 /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3P9P1 /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3Q95P /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3QGV5 /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3R74X /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3SZCM /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3WHE7]}}
[Debug] Collecting S.M.A.R.T. counters, device: /dev/disk/by-id/scsi-SATA_PNY_CS900_120GB_PNY0520228055010E219
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_PNY_CS900_120GB_PNY0520228055010E219: , PNY CS900 120GB SSD
[Debug] Collecting S.M.A.R.T. counters, device: /dev/disk/by-id/scsi-SATA_PNY_CS900_120GB_PNY0520228055010E21B
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_PNY_CS900_120GB_PNY0520228055010E21B: , PNY CS900 120GB SSD
[Debug] Collecting S.M.A.R.T. counters, device: /dev/disk/by-id/scsi-SATA_ST10000NM0016-1T_ZA2CZA35
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_ST10000NM0016-1T_ZA2CZA35: Seagate Enterprise Capacity 3.5 HDD, ST10000NM0016-1TT101
[Debug] Collecting S.M.A.R.T. counters, device: /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3NY2V
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3NY2V: Seagate Exos X14, ST12000NM0008-2H3101
[Debug] Collecting S.M.A.R.T. counters, device: /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3P3E3
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3P3E3: Seagate Exos X14, ST12000NM0008-2H3101
[Debug] Collecting S.M.A.R.T. counters, device: /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3P9P1
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3P9P1: Seagate Exos X14, ST12000NM0008-2H3101
[Debug] Collecting S.M.A.R.T. counters, device: /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3Q95P
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3Q95P: Seagate Exos X14, ST12000NM0008-2H3101
[Debug] Collecting S.M.A.R.T. counters, device: /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3QGV5
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3QGV5: Seagate Exos X14, ST12000NM0008-2H3101
[Debug] Collecting S.M.A.R.T. counters, device: /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3R74X
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3R74X: Seagate Exos X14, ST12000NM0008-2H3101
[Debug] Collecting S.M.A.R.T. counters, device: /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3SZCM
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3SZCM: Seagate Exos X14, ST12000NM0008-2H3101
[Debug] Collecting S.M.A.R.T. counters, device: /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3WHE7
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3WHE7: Seagate Exos X14, ST12000NM0008-2H3101
[Info] Starting on [::]:9666/metrics
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_PNY_CS900_120GB_PNY0520228055010E219: , PNY CS900 120GB SSD
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_PNY_CS900_120GB_PNY0520228055010E21B: , PNY CS900 120GB SSD
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_ST10000NM0016-1T_ZA2CZA35: Seagate Enterprise Capacity 3.5 HDD, ST10000NM0016-1TT101
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3NY2V: Seagate Exos X14, ST12000NM0008-2H3101
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3P3E3: Seagate Exos X14, ST12000NM0008-2H3101
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3P9P1: Seagate Exos X14, ST12000NM0008-2H3101
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3Q95P: Seagate Exos X14, ST12000NM0008-2H3101
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3QGV5: Seagate Exos X14, ST12000NM0008-2H3101
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3R74X: Seagate Exos X14, ST12000NM0008-2H3101
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3SZCM: Seagate Exos X14, ST12000NM0008-2H3101
[Verbose] Collecting metrics from /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3WHE7: Seagate Exos X14, ST12000NM0008-2H3101

I ran smartctl --json --xall /dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3P9P1 > out.json
out.json:
{
  "json_format_version": [
    1,
    0
  ],
  "smartctl": {
    "version": [
      7,
      1
    ],
    "svn_revision": "5022",
    "platform_info": "x86_64-linux-5.4.0-40-generic",
    "build_info": "(local build)",
    "argv": [
      "smartctl",
      "--json",
      "--xall",
      "/dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3P9P1"
    ],
    "exit_status": 0
  },
  "device": {
    "name": "/dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3P9P1",
    "info_name": "/dev/disk/by-id/scsi-SATA_ST12000NM0008-2H_ZHZ3P9P1 [SAT]",
    "type": "sat",
    "protocol": "ATA"
  },
  "model_family": "Seagate Exos X14",
  "model_name": "ST12000NM0008-2H3101",
  "serial_number": "ZHZ3P9P1",
  "wwn": {
    "naa": 5,
    "oui": 3152,
    "id": 3265897365
  },
  "firmware_version": "SN02",
  "user_capacity": {
    "blocks": 23437770752,
    "bytes": 12000138625024
  },
  "logical_block_size": 512,
  "physical_block_size": 4096,
  "rotation_rate": 7200,
  "form_factor": {
    "ata_value": 2,
    "name": "3.5 inches"
  },
  "in_smartctl_database": true,
  "ata_version": {
    "string": "ACS-4 T13/BSR INCITS 529 revision 5",
    "major_value": 4064,
    "minor_value": 94
  },
  "sata_version": {
    "string": "SATA 3.3",
    "value": 511
  },
  "interface_speed": {
    "max": {
      "sata_value": 14,
      "string": "6.0 Gb/s",
      "units_per_second": 60,
      "bits_per_unit": 100000000
    },
    "current": {
      "sata_value": 3,
      "string": "6.0 Gb/s",
      "units_per_second": 60,
      "bits_per_unit": 100000000
    }
  },
  "local_time": {
    "time_t": 1593997637,
    "asctime": "Mon Jul  6 01:07:17 2020 UTC"
  },
  "read_lookahead": {
    "enabled": true
  },
  "write_cache": {
    "enabled": true
  },
  "ata_dsn": {
    "enabled": false
  },
  "ata_security": {
    "state": 33,
    "string": "Disabled, NOT FROZEN [SEC1]",
    "enabled": false,
    "frozen": false
  },
  "smart_status": {
    "passed": true
  },
  "ata_smart_data": {
    "offline_data_collection": {
      "status": {
        "value": 130,
        "string": "was completed without error",
        "passed": true
      },
      "completion_seconds": 575
    },
    "self_test": {
      "status": {
        "value": 0,
        "string": "completed without error",
        "passed": true
      },
      "polling_minutes": {
        "short": 1,
        "extended": 1174,
        "conveyance": 2
      }
    },
    "capabilities": {
      "values": [
        123,
        3
      ],
      "exec_offline_immediate_supported": true,
      "offline_is_aborted_upon_new_cmd": false,
      "offline_surface_scan_supported": true,
      "self_tests_supported": true,
      "conveyance_self_test_supported": true,
      "selective_self_test_supported": true,
      "attribute_autosave_enabled": true,
      "error_logging_supported": true,
      "gp_logging_supported": true
    }
  },
  "ata_sct_capabilities": {
    "value": 28861,
    "error_recovery_control_supported": true,
    "feature_control_supported": true,
    "data_table_supported": true
  },
  "ata_smart_attributes": {
    "revision": 10,
    "table": [
      {
        "id": 1,
        "name": "Raw_Read_Error_Rate",
        "value": 81,
        "worst": 64,
        "thresh": 44,
        "when_failed": "",
        "flags": {
          "value": 15,
          "string": "POSR-- ",
          "prefailure": true,
          "updated_online": true,
          "performance": true,
          "error_rate": true,
          "event_count": false,
          "auto_keep": false
        },
        "raw": {
          "value": 124098219,
          "string": "124098219"
        }
      },
      {
        "id": 3,
        "name": "Spin_Up_Time",
        "value": 98,
        "worst": 98,
        "thresh": 0,
        "when_failed": "",
        "flags": {
          "value": 3,
          "string": "PO---- ",
          "prefailure": true,
          "updated_online": true,
          "performance": false,
          "error_rate": false,
          "event_count": false,
          "auto_keep": false
        },
        "raw": {
          "value": 0,
          "string": "0"
        }
      },
      {
        "id": 4,
        "name": "Start_Stop_Count",
        "value": 100,
        "worst": 100,
        "thresh": 20,
        "when_failed": "",
        "flags": {
          "value": 50,
          "string": "-O--CK ",
          "prefailure": false,
          "updated_online": true,
          "performance": false,
          "error_rate": false,
          "event_count": true,
          "auto_keep": true
        },
        "raw": {
          "value": 3,
          "string": "3"
        }
      },
      {
        "id": 5,
        "name": "Reallocated_Sector_Ct",
        "value": 100,
        "worst": 100,
        "thresh": 10,
        "when_failed": "",
        "flags": {
          "value": 51,
          "string": "PO--CK ",
          "prefailure": true,
          "updated_online": true,
          "performance": false,
          "error_rate": false,
          "event_count": true,
          "auto_keep": true
        },
        "raw": {
          "value": 0,
          "string": "0"
        }
      },
      {
        "id": 7,
        "name": "Seek_Error_Rate",
        "value": 74,
        "worst": 60,
        "thresh": 45,
        "when_failed": "",
        "flags": {
          "value": 15,
          "string": "POSR-- ",
          "prefailure": true,
          "updated_online": true,
          "performance": true,
          "error_rate": true,
          "event_count": false,
          "auto_keep": false
        },
        "raw": {
          "value": 24153386,
          "string": "24153386"
        }
      },
      {
        "id": 9,
        "name": "Power_On_Hours",
        "value": 99,
        "worst": 99,
        "thresh": 0,
        "when_failed": "",
        "flags": {
          "value": 50,
          "string": "-O--CK ",
          "prefailure": false,
          "updated_online": true,
          "performance": false,
          "error_rate": false,
          "event_count": true,
          "auto_keep": true
        },
        "raw": {
          "value": 1276,
          "string": "1276"
        }
      },
      {
        "id": 10,
        "name": "Spin_Retry_Count",
        "value": 100,
        "worst": 100,
        "thresh": 97,
        "when_failed": "",
        "flags": {
          "value": 19,
          "string": "PO--C- ",
          "prefailure": true,
          "updated_online": true,
          "performance": false,
          "error_rate": false,
          "event_count": true,
          "auto_keep": false
        },
        "raw": {
          "value": 0,
          "string": "0"
        }
      },
      {
        "id": 12,
        "name": "Power_Cycle_Count",
        "value": 100,
        "worst": 100,
        "thresh": 20,
        "when_failed": "",
        "flags": {
          "value": 50,
          "string": "-O--CK ",
          "prefailure": false,
          "updated_online": true,
          "performance": false,
          "error_rate": false,
          "event_count": true,
          "auto_keep": true
        },
        "raw": {
          "value": 3,
          "string": "3"
        }
      },
      {
        "id": 18,
        "name": "Unknown_Attribute",
        "value": 100,
        "worst": 100,
        "thresh": 50,
        "when_failed": "",
        "flags": {
          "value": 11,
          "string": "PO-R-- ",
          "prefailure": true,
          "updated_online": true,
          "performance": false,
          "error_rate": true,
          "event_count": false,
          "auto_keep": false
        },
        "raw": {
          "value": 0,
          "string": "0"
        }
      },
      {
        "id": 187,
        "name": "Reported_Uncorrect",
        "value": 100,
        "worst": 100,
        "thresh": 0,
        "when_failed": "",
        "flags": {
          "value": 50,
          "string": "-O--CK ",
          "prefailure": false,
          "updated_online": true,
          "performance": false,
          "error_rate": false,
          "event_count": true,
          "auto_keep": true
        },
        "raw": {
          "value": 0,
          "string": "0"
        }
      },
      {
        "id": 188,
        "name": "Command_Timeout",
        "value": 100,
        "worst": 100,
        "thresh": 0,
        "when_failed": "",
        "flags": {
          "value": 50,
          "string": "-O--CK ",
          "prefailure": false,
          "updated_online": true,
          "performance": false,
          "error_rate": false,
          "event_count": true,
          "auto_keep": true
        },
        "raw": {
          "value": 0,
          "string": "0"
        }
      },
      {
        "id": 190,
        "name": "Airflow_Temperature_Cel",
        "value": 58,
        "worst": 57,
        "thresh": 40,
        "when_failed": "",
        "flags": {
          "value": 34,
          "string": "-O---K ",
          "prefailure": false,
          "updated_online": true,
          "performance": false,
          "error_rate": false,
          "event_count": false,
          "auto_keep": true
        },
        "raw": {
          "value": 723386410,
          "string": "42 (Min/Max 30/43)"
        }
      },
      {
        "id": 192,
        "name": "Power-Off_Retract_Count",
        "value": 100,
        "worst": 100,
        "thresh": 0,
        "when_failed": "",
        "flags": {
          "value": 50,
          "string": "-O--CK ",
          "prefailure": false,
          "updated_online": true,
          "performance": false,
          "error_rate": false,
          "event_count": true,
          "auto_keep": true
        },
        "raw": {
          "value": 5,
          "string": "5"
        }
      },
      {
        "id": 193,
        "name": "Load_Cycle_Count",
        "value": 100,
        "worst": 100,
        "thresh": 0,
        "when_failed": "",
        "flags": {
          "value": 50,
          "string": "-O--CK ",
          "prefailure": false,
          "updated_online": true,
          "performance": false,
          "error_rate": false,
          "event_count": true,
          "auto_keep": true
        },
        "raw": {
          "value": 452,
          "string": "452"
        }
      },
      {
        "id": 194,
        "name": "Temperature_Celsius",
        "value": 42,
        "worst": 43,
        "thresh": 0,
        "when_failed": "",
        "flags": {
          "value": 34,
          "string": "-O---K ",
          "prefailure": false,
          "updated_online": true,
          "performance": false,
          "error_rate": false,
          "event_count": false,
          "auto_keep": true
        },
        "raw": {
          "value": 120259084330,
          "string": "42 (0 28 0 0 0)"
        }
      },
      {
        "id": 195,
        "name": "Hardware_ECC_Recovered",
        "value": 16,
        "worst": 14,
        "thresh": 0,
        "when_failed": "",
        "flags": {
          "value": 26,
          "string": "-O-RC- ",
          "prefailure": false,
          "updated_online": true,
          "performance": false,
          "error_rate": true,
          "event_count": true,
          "auto_keep": false
        },
        "raw": {
          "value": 124098219,
          "string": "124098219"
        }
      },
      {
        "id": 197,
        "name": "Current_Pending_Sector",
        "value": 100,
        "worst": 100,
        "thresh": 0,
        "when_failed": "",
        "flags": {
          "value": 18,
          "string": "-O--C- ",
          "prefailure": false,
          "updated_online": true,
          "performance": false,
          "error_rate": false,
          "event_count": true,
          "auto_keep": false
        },
        "raw": {
          "value": 0,
          "string": "0"
        }
      },
      {
        "id": 198,
        "name": "Offline_Uncorrectable",
        "value": 100,
        "worst": 100,
        "thresh": 0,
        "when_failed": "",
        "flags": {
          "value": 16,
          "string": "----C- ",
          "prefailure": false,
          "updated_online": false,
          "performance": false,
          "error_rate": false,
          "event_count": true,
          "auto_keep": false
        },
        "raw": {
          "value": 0,
          "string": "0"
        }
      },
      {
        "id": 199,
        "name": "UDMA_CRC_Error_Count",
        "value": 200,
        "worst": 200,
        "thresh": 0,
        "when_failed": "",
        "flags": {
          "value": 62,
          "string": "-OSRCK ",
          "prefailure": false,
          "updated_online": true,
          "performance": true,
          "error_rate": true,
          "event_count": true,
          "auto_keep": true
        },
        "raw": {
          "value": 0,
          "string": "0"
        }
      },
      {
        "id": 200,
        "name": "Multi_Zone_Error_Rate",
        "value": 100,
        "worst": 100,
        "thresh": 1,
        "when_failed": "",
        "flags": {
          "value": 35,
          "string": "PO---K ",
          "prefailure": true,
          "updated_online": true,
          "performance": false,
          "error_rate": false,
          "event_count": false,
          "auto_keep": true
        },
        "raw": {
          "value": 0,
          "string": "0"
        }
      },
      {
        "id": 240,
        "name": "Head_Flying_Hours",
        "value": 100,
        "worst": 253,
        "thresh": 0,
        "when_failed": "",
        "flags": {
          "value": 0,
          "string": "------ ",
          "prefailure": false,
          "updated_online": false,
          "performance": false,
          "error_rate": false,
          "event_count": false,
          "auto_keep": false
        },
        "raw": {
          "value": 12903271463125164,
          "string": "172h+50m+04.277s"
        }
      },
      {
        "id": 241,
        "name": "Total_LBAs_Written",
        "value": 100,
        "worst": 253,
        "thresh": 0,
        "when_failed": "",
        "flags": {
          "value": 0,
          "string": "------ ",
          "prefailure": false,
          "updated_online": false,
          "performance": false,
          "error_rate": false,
          "event_count": false,
          "auto_keep": false
        },
        "raw": {
          "value": 22822792513,
          "string": "22822792513"
        }
      },
      {
        "id": 242,
        "name": "Total_LBAs_Read",
        "value": 100,
        "worst": 253,
        "thresh": 0,
        "when_failed": "",
        "flags": {
          "value": 0,
          "string": "------ ",
          "prefailure": false,
          "updated_online": false,
          "performance": false,
          "error_rate": false,
          "event_count": false,
          "auto_keep": false
        },
        "raw": {
          "value": 6432600,
          "string": "6432600"
        }
      }
    ]
  },
  "power_on_time": {
    "hours": 1276
  },
  "power_cycle_count": 3,
  "temperature": {
    "current": 42,
    "power_cycle_min": 30,
    "power_cycle_max": 43,
    "lifetime_min": 0,
    "lifetime_max": 43,
    "op_limit_min": 5,
    "op_limit_max": 60,
    "limit_min": 5,
    "limit_max": 60,
    "lifetime_over_limit_minutes": 0,
    "lifetime_under_limit_minutes": 7230
  },
  "ata_log_directory": {
    "gp_dir_version": 1,
    "smart_dir_version": 1,
    "smart_dir_multi_sector": true,
    "table": [
      {
        "address": 0,
        "name": "Log Directory",
        "read": true,
        "write": false,
        "gp_sectors": 1,
        "smart_sectors": 1
      },
      {
        "address": 1,
        "name": "Summary SMART error log",
        "read": true,
        "write": false,
        "smart_sectors": 1
      },
      {
        "address": 2,
        "name": "Comprehensive SMART error log",
        "read": true,
        "write": false,
        "smart_sectors": 5
      },
      {
        "address": 3,
        "name": "Ext. Comprehensive SMART error log",
        "read": true,
        "write": false,
        "gp_sectors": 5
      },
      {
        "address": 4,
        "name": "Device Statistics log",
        "read": true,
        "write": false,
        "gp_sectors": 256,
        "smart_sectors": 8
      },
      {
        "address": 6,
        "name": "SMART self-test log",
        "read": true,
        "write": false,
        "smart_sectors": 1
      },
      {
        "address": 7,
        "name": "Extended self-test log",
        "read": true,
        "write": false,
        "gp_sectors": 1
      },
      {
        "address": 8,
        "name": "Power Conditions log",
        "read": true,
        "write": false,
        "gp_sectors": 2
      },
      {
        "address": 9,
        "name": "Selective self-test log",
        "read": true,
        "write": true,
        "smart_sectors": 1
      },
      {
        "address": 10,
        "name": "Device Statistics Notification",
        "read": true,
        "write": true,
        "gp_sectors": 8
      },
      {
        "address": 12,
        "name": "Pending Defects log",
        "read": true,
        "write": false,
        "gp_sectors": 2048
      },
      {
        "address": 16,
        "name": "NCQ Command Error log",
        "read": true,
        "write": false,
        "gp_sectors": 1
      },
      {
        "address": 17,
        "name": "SATA Phy Event Counters log",
        "read": true,
        "write": false,
        "gp_sectors": 1
      },
      {
        "address": 19,
        "name": "SATA NCQ Send and Receive log",
        "read": true,
        "write": false,
        "gp_sectors": 1
      },
      {
        "address": 21,
        "name": "Rebuild Assist log",
        "read": true,
        "write": true,
        "gp_sectors": 1
      },
      {
        "address": 33,
        "name": "Write stream error log",
        "read": true,
        "write": false,
        "gp_sectors": 1
      },
      {
        "address": 34,
        "name": "Read stream error log",
        "read": true,
        "write": false,
        "gp_sectors": 1
      },
      {
        "address": 36,
        "name": "Current Device Internal Status Data log",
        "read": true,
        "write": false,
        "gp_sectors": 768
      },
      {
        "address": 47,
        "name": "Set Sector Configuration",
        "gp_sectors": 1
      },
      {
        "address": 48,
        "name": "IDENTIFY DEVICE data log",
        "read": true,
        "write": false,
        "gp_sectors": 9,
        "smart_sectors": 9
      },
      {
        "address": 128,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 129,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 130,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 131,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 132,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 133,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 134,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 135,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 136,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 137,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 138,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 139,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 140,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 141,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 142,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 143,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 144,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 145,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 146,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 147,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 148,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 149,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 150,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 151,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 152,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 153,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 154,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 155,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 156,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 157,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 158,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 159,
        "name": "Host vendor specific log",
        "read": true,
        "write": true,
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 161,
        "name": "Device vendor specific log",
        "gp_sectors": 24,
        "smart_sectors": 24
      },
      {
        "address": 162,
        "name": "Device vendor specific log",
        "gp_sectors": 16320
      },
      {
        "address": 164,
        "name": "Device vendor specific log",
        "gp_sectors": 160,
        "smart_sectors": 160
      },
      {
        "address": 166,
        "name": "Device vendor specific log",
        "gp_sectors": 192
      },
      {
        "address": 168,
        "name": "Device vendor specific log",
        "gp_sectors": 136,
        "smart_sectors": 136
      },
      {
        "address": 169,
        "name": "Device vendor specific log",
        "gp_sectors": 136,
        "smart_sectors": 136
      },
      {
        "address": 171,
        "name": "Device vendor specific log",
        "gp_sectors": 1
      },
      {
        "address": 173,
        "name": "Device vendor specific log",
        "gp_sectors": 16
      },
      {
        "address": 190,
        "name": "Device vendor specific log",
        "gp_sectors": 65535
      },
      {
        "address": 191,
        "name": "Device vendor specific log",
        "gp_sectors": 65535
      },
      {
        "address": 193,
        "name": "Device vendor specific log",
        "gp_sectors": 8,
        "smart_sectors": 8
      },
      {
        "address": 195,
        "name": "Device vendor specific log",
        "gp_sectors": 32,
        "smart_sectors": 32
      },
      {
        "address": 201,
        "name": "Device vendor specific log",
        "gp_sectors": 8,
        "smart_sectors": 8
      },
      {
        "address": 202,
        "name": "Device vendor specific log",
        "gp_sectors": 16,
        "smart_sectors": 16
      },
      {
        "address": 209,
        "name": "Device vendor specific log",
        "gp_sectors": 336
      },
      {
        "address": 210,
        "name": "Device vendor specific log",
        "gp_sectors": 10000
      },
      {
        "address": 212,
        "name": "Device vendor specific log",
        "gp_sectors": 2048
      },
      {
        "address": 218,
        "name": "Device vendor specific log",
        "gp_sectors": 1,
        "smart_sectors": 1
      },
      {
        "address": 224,
        "name": "SCT Command/Status",
        "read": true,
        "write": true,
        "gp_sectors": 1,
        "smart_sectors": 1
      },
      {
        "address": 225,
        "name": "SCT Data Transfer",
        "read": true,
        "write": true,
        "gp_sectors": 1,
        "smart_sectors": 1
      }
    ]
  },
  "ata_smart_error_log": {
    "extended": {
      "revision": 1,
      "sectors": 5,
      "count": 0
    }
  },
  "ata_smart_self_test_log": {
    "extended": {
      "revision": 1,
      "sectors": 1,
      "count": 0
    }
  },
  "ata_smart_selective_self_test_log": {
    "revision": 1,
    "table": [
      {
        "lba_min": 0,
        "lba_max": 0,
        "status": {
          "value": 0,
          "string": "Not_testing"
        }
      },
      {
        "lba_min": 0,
        "lba_max": 0,
        "status": {
          "value": 0,
          "string": "Not_testing"
        }
      },
      {
        "lba_min": 0,
        "lba_max": 0,
        "status": {
          "value": 0,
          "string": "Not_testing"
        }
      },
      {
        "lba_min": 0,
        "lba_max": 0,
        "status": {
          "value": 0,
          "string": "Not_testing"
        }
      },
      {
        "lba_min": 0,
        "lba_max": 0,
        "status": {
          "value": 0,
          "string": "Not_testing"
        }
      }
    ],
    "flags": {
      "value": 0,
      "remainder_scan_enabled": false
    },
    "power_up_scan_resume_minutes": 0
  },
  "ata_sct_status": {
    "format_version": 3,
    "sct_version": 522,
    "device_state": {
      "value": 0,
      "string": "Active"
    },
    "temperature": {
      "current": 42,
      "power_cycle_min": 30,
      "power_cycle_max": 43,
      "lifetime_min": 26,
      "lifetime_max": 43,
      "under_limit_count": 0,
      "over_limit_count": 4
    }
  },
  "ata_sct_temperature_history": {
    "version": 2,
    "sampling_period_minutes": 3,
    "logging_interval_minutes": 59,
    "temperature": {
      "op_limit_min": 10,
      "op_limit_max": 40,
      "limit_min": 5,
      "limit_max": 60
    },
    "size": 128,
    "index": 26,
    "table": [
      33,
      34,
      35,
      36,
      37,
      37,
      37,
      37,
      37,
      37,
      37,
      36,
      36,
      35,
      35,
      34,
      34,
      33,
      33,
      33,
      33,
      32,
      32,
      33,
      34,
      35,
      36,
      37,
      37,
      38,
      38,
      38,
      38,
      37,
      37,
      37,
      36,
      36,
      36,
      35,
      35,
      35,
      34,
      34,
      34,
      34,
      33,
      34,
      35,
      36,
      37,
      38,
      38,
      38,
      38,
      38,
      38,
      38,
      38,
      37,
      37,
      37,
      36,
      36,
      35,
      35,
      35,
      34,
      34,
      34,
      34,
      34,
      35,
      36,
      37,
      38,
      39,
      39,
      39,
      39,
      39,
      39,
      38,
      38,
      39,
      39,
      38,
      38,
      37,
      37,
      36,
      36,
      35,
      35,
      35,
      35,
      36,
      37,
      38,
      39,
      40,
      40,
      40,
      40,
      40,
      40,
      39,
      39,
      38,
      38,
      38,
      37,
      37,
      36,
      36,
      36,
      35,
      35,
      35,
      35,
      36,
      37,
      38,
      39,
      40,
      40,
      41,
      43
    ]
  },
  "ata_sct_erc": {
    "read": {
      "enabled": true,
      "deciseconds": 100
    },
    "write": {
      "enabled": true,
      "deciseconds": 100
    }
  },
  "ata_device_statistics": {
    "pages": [
      {
        "number": 1,
        "name": "General Statistics",
        "revision": 1,
        "table": [
          {
            "offset": 8,
            "name": "Lifetime Power-On Resets",
            "size": 4,
            "value": 3,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 16,
            "name": "Power-on Hours",
            "size": 4,
            "value": 1276,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 24,
            "name": "Logical Sectors Written",
            "size": 6,
            "value": 22822792513,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 32,
            "name": "Number of Write Commands",
            "size": 6,
            "value": 71485685,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 40,
            "name": "Logical Sectors Read",
            "size": 6,
            "value": 6432600,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 48,
            "name": "Number of Read Commands",
            "size": 6,
            "value": 202309,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 56,
            "name": "Date and Time TimeStamp",
            "size": 6,
            "flags": {
              "value": 128,
              "string": "---- ",
              "valid": false,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          }
        ]
      },
      {
        "number": 3,
        "name": "Rotating Media Statistics",
        "revision": 1,
        "table": [
          {
            "offset": 8,
            "name": "Spindle Motor Power-on Hours",
            "size": 4,
            "value": 1157,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 16,
            "name": "Head Flying Hours",
            "size": 4,
            "value": 203,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 24,
            "name": "Head Load Events",
            "size": 4,
            "value": 452,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 32,
            "name": "Number of Reallocated Logical Sectors",
            "size": 4,
            "value": 0,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 40,
            "name": "Read Recovery Attempts",
            "size": 4,
            "value": 0,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 48,
            "name": "Number of Mechanical Start Failures",
            "size": 4,
            "value": 0,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 56,
            "name": "Number of Realloc. Candidate Logical Sectors",
            "size": 4,
            "value": 0,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 64,
            "name": "Number of High Priority Unload Events",
            "size": 4,
            "value": 5,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          }
        ]
      },
      {
        "number": 4,
        "name": "General Errors Statistics",
        "revision": 1,
        "table": [
          {
            "offset": 8,
            "name": "Number of Reported Uncorrectable Errors",
            "size": 4,
            "value": 0,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 16,
            "name": "Resets Between Cmd Acceptance and Completion",
            "size": 4,
            "value": 0,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 24,
            "name": "Physical Element Status Changed",
            "size": 4,
            "value": 0,
            "flags": {
              "value": 208,
              "string": "V-D- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": true,
              "monitored_condition_met": false
            }
          }
        ]
      },
      {
        "number": 5,
        "name": "Temperature Statistics",
        "revision": 1,
        "table": [
          {
            "offset": 8,
            "name": "Current Temperature",
            "size": 1,
            "value": 42,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 16,
            "name": "Average Short Term Temperature",
            "size": 1,
            "value": 38,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 24,
            "name": "Average Long Term Temperature",
            "size": 1,
            "value": 30,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 32,
            "name": "Highest Temperature",
            "size": 1,
            "value": 43,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 40,
            "name": "Lowest Temperature",
            "size": 1,
            "value": 0,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 48,
            "name": "Highest Average Short Term Temperature",
            "size": 1,
            "value": 40,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 56,
            "name": "Lowest Average Short Term Temperature",
            "size": 1,
            "value": 0,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 64,
            "name": "Highest Average Long Term Temperature",
            "size": 1,
            "value": 30,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 72,
            "name": "Lowest Average Long Term Temperature",
            "size": 1,
            "value": 29,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 80,
            "name": "Time in Over-Temperature",
            "size": 4,
            "value": 0,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 88,
            "name": "Specified Maximum Operating Temperature",
            "size": 1,
            "value": 60,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 96,
            "name": "Time in Under-Temperature",
            "size": 4,
            "value": 7230,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 104,
            "name": "Specified Minimum Operating Temperature",
            "size": 1,
            "value": 5,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          }
        ]
      },
      {
        "number": 6,
        "name": "Transport Statistics",
        "revision": 1,
        "table": [
          {
            "offset": 8,
            "name": "Number of Hardware Resets",
            "size": 4,
            "value": 1,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 16,
            "name": "Number of ASR Events",
            "size": 4,
            "value": 0,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 24,
            "name": "Number of Interface CRC Errors",
            "size": 4,
            "value": 0,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          }
        ]
      },
      {
        "number": 255,
        "name": "Vendor Specific Statistics",
        "revision": 1,
        "table": [
          {
            "offset": 8,
            "name": "Vendor Specific",
            "size": 7,
            "value": 0,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 16,
            "name": "Vendor Specific",
            "size": 7,
            "value": 0,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 24,
            "name": "Vendor Specific",
            "size": 7,
            "value": 0,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          }
        ]
      }
    ]
  },
  "ata_pending_defects_log": {
    "size": 65535,
    "count": 0
  },
  "sata_phy_event_counters": {
    "table": [
      {
        "id": 10,
        "name": "Device-to-host register FISes sent due to a COMRESET",
        "size": 2,
        "value": 2,
        "overflow": false
      },
      {
        "id": 1,
        "name": "Command failed due to ICRC error",
        "size": 2,
        "value": 0,
        "overflow": false
      },
      {
        "id": 3,
        "name": "R_ERR response for device-to-host data FIS",
        "size": 2,
        "value": 0,
        "overflow": false
      },
      {
        "id": 4,
        "name": "R_ERR response for host-to-device data FIS",
        "size": 2,
        "value": 0,
        "overflow": false
      },
      {
        "id": 6,
        "name": "R_ERR response for device-to-host non-data FIS",
        "size": 2,
        "value": 0,
        "overflow": false
      },
      {
        "id": 7,
        "name": "R_ERR response for host-to-device non-data FIS",
        "size": 2,
        "value": 0,
        "overflow": false
      }
    ],
    "reset": false
  }
}

The problem is that smartctl reports some fields for which the smartctl_exporter extracts duplicate name and label values. Notably (from the above output):
      {
        "number": 255,
        "name": "Vendor Specific Statistics",
        "revision": 1,
        "table": [
          {
            "offset": 8,
            "name": "Vendor Specific",
            "size": 7,
            "value": 0,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 16,
            "name": "Vendor Specific",
            "size": 7,
            "value": 0,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          },
          {
            "offset": 24,
            "name": "Vendor Specific",
            "size": 7,
            "value": 0,
            "flags": {
              "value": 192,
              "string": "V--- ",
              "valid": true,
              "normalized": false,
              "supports_dsn": false,
              "monitored_condition_met": false
            }
          }
        ]
      }

Here the metric name is smartctl_device_statistics and label is <name:"statistic_name" value:"Vendor Specific" > label:<name:"statistic_table" value:"Vendor Specific Statistics" > gauge:<value:0 > for all three fields. You can see a warning generated for the latter two fields.

I am facing the same issue in latest release 0.7.0. Let me know if you need more logs in order to debug this.

authors: AdolarMezga Sheridan keliansb lahwaacz sneak


text:
Document that target_label can do regex replacement

Fixes #602

:+1:

source: gharchive/pull-request

repo: prometheus/docs

license_type: permissive

license_source: bigquery


text:
Parse /sys/class/drm/cardN/device/ stats

This PR adds support for reading card stats exposed by drm.
Only amdgpu driver currently exposes these stats but it should be easy to add more driver support if required.
FreeBSD uses the same driver through kms-drm but it does not expose the attributes we are interested in through niether sysctl or linsysfs.

Hopefully other GPU vendors also add similar stats.

authors: SuperQ siavashs

repo: prometheus/procfs


text:
stacked graphs are not deep linked anymore

What did you do?
If you create a query with multiple results and switch to the stacked graph, it will reset to the unstacked lines every time you:

click refresh
open a link
click the execute button

in the previously deployed version (v2.48.1), it still worked correctly (as it did the last years). Could this PR be responsible for the regression? https://github.com/prometheus/prometheus/pull/13096
What did you expect to see?
stacked lines to be remembered, like in the previous version of prometheus
What did you see instead? Under which circumstances?
unstacked lines
System information
No response
Prometheus version
Build Information
Version	2.49.1
Revision	43e14844a33b65e2a396e3944272af8b3a494071
Branch	HEAD
BuildUser	root@6d5f4c649d25
BuildDate	20240115-16:58:43
GoVersion	go1.21.6

Prometheus configuration file
No response
Alertmanager version
No response
Alertmanager configuration file
No response
Logs
No response

CC @Loori-R @juliusv @beorn7 as you were involved in #13096

closed with https://github.com/prometheus/prometheus/pull/13460

source: gharchive/issue

authors: Nexucis peterhoneder raoulbhatia

repo: prometheus/prometheus

license_source: github-api


text:
tsdb Panic when reading chunk

Bug Report
What did you do?
Ran prometheus
What did you expect to see?
It run
What did you see instead? Under which circumstances?
Exception on startup: panic: runtime error: index out of range (looks similar to: https://github.com/prometheus/tsdb/issues/251)
Environment

System information:

/prometheus $ uname -srm
Linux 4.15.3-1.el7.elrepo.x86_64 x86_64

Prometheus version:

/prometheus $ prometheus --version
prometheus, version 2.2.1 (branch: HEAD, revision: bc6058c81272a8d938c05e75607371284236aadc)
  build user:       root@149e5b3f0829
  build date:       20180314-14:15:45
  go version:       go1.10

Logs:

level=info ts=2018-05-01T12:35:26.455261083Z caller=main.go:220 msg="Starting Prometheus" version="(version=2.2.1, branch=HEAD, revision=bc6058c81272a8d938c05e75607371284236aadc)"
level=info ts=2018-05-01T12:35:26.45538143Z caller=main.go:221 build_context="(go=go1.10, user=root@149e5b3f0829, date=20180314-14:15:45)"
level=info ts=2018-05-01T12:35:26.455414873Z caller=main.go:222 host_details="(Linux 4.15.3-1.el7.elrepo.x86_64 #1 SMP Mon Feb 12 06:46:25 EST 2018 x86_64 prometheus-59bf4d7cf-sxsk5 (none))"
level=info ts=2018-05-01T12:35:26.455427991Z caller=main.go:223 fd_limits="(soft=65536, hard=65536)"
level=info ts=2018-05-01T12:35:26.459927519Z caller=web.go:382 component=web msg="Start listening for connections" address=0.0.0.0:9090
level=info ts=2018-05-01T12:35:26.45947935Z caller=main.go:504 msg="Starting TSDB ..."
level=error ts=2018-05-01T12:36:48.215666256Z caller=wal.go:275 component=tsdb msg="WAL corruption detected; truncating" err="unexpected CRC32 checksum 6cafd16a, want 0" file=/data/prometheus/wal/033372 pos=75965705
level=info ts=2018-05-01T12:36:48.950404399Z caller=main.go:514 msg="TSDB started"
level=info ts=2018-05-01T12:36:48.950585155Z caller=main.go:588 msg="Loading configuration file" filename=/etc/prometheus/config.yaml
level=info ts=2018-05-01T12:36:48.96267101Z caller=kubernetes.go:191 component="discovery manager scrape" discovery=k8s msg="Using pod service account via in-cluster config"
level=info ts=2018-05-01T12:36:48.965331005Z caller=kubernetes.go:191 component="discovery manager scrape" discovery=k8s msg="Using pod service account via in-cluster config"
level=info ts=2018-05-01T12:36:48.966139809Z caller=kubernetes.go:191 component="discovery manager scrape" discovery=k8s msg="Using pod service account via in-cluster config"
level=info ts=2018-05-01T12:36:48.966845893Z caller=kubernetes.go:191 component="discovery manager scrape" discovery=k8s msg="Using pod service account via in-cluster config"
level=info ts=2018-05-01T12:36:48.96989835Z caller=main.go:491 msg="Server is ready to receive web requests."
level=info ts=2018-05-01T12:37:48.996924066Z caller=compact.go:393 component=tsdb msg="compact blocks" count=4 mint=1524960000000 maxt=1524981600000
panic: runtime error: index out of range

goroutine 403 [running]:
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/chunks.(*Reader).Chunk(0xc75cb30c40, 0x13e, 0x1c56c80, 0xc5b252e0f0, 0x0, 0x0)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/chunks/chunks.go:365 +0x3f8
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*compactionSeriesSet).Next(0xc75bbc0dc0, 0xc763d9c770)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:691 +0x223
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.newCompactionMerger(0x1c47a80, 0xc75bbc0dc0, 0x1c47a80, 0xc75bbc0e60, 0xc7687230a0, 0x1c51880, 0xc7687230a0)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:733 +0x8f
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*LeveledCompactor).populateBlock(0xc420d93220, 0xc75cb30e80, 0x4, 0x4, 0xc763d9c540, 0x1c56cc0, 0xc768766000, 0x1c3e680, 0xc7671fe640, 0x0, ...)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:537 +0x5bf
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*LeveledCompactor).write(0xc420d93220, 0x7ffc588f58e3, 0x10, 0xc763d9c540, 0xc75cb30e80, 0x4, 0x4, 0x0, 0x0)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:441 +0x5dd
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*LeveledCompactor).Compact(0xc420d93220, 0x7ffc588f58e3, 0x10, 0xc75cb30b40, 0x4, 0x4, 0x3023447db51b6301, 0x6431175cbef51e81, 0x0, 0x0)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:339 +0x693
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*DB).compact(0xc420c7a0c0, 0x0, 0x0, 0x0)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:419 +0x400
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*DB).run(0xc420c7a0c0)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:279 +0x2cc
created by github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.Open
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:243 +0x5ba

Apologies if this is in the wrong repo. Any advice on recovering from this error without removing all tsdb files would be appreciated.

had  quick look and it seems some corruption in the chunk so it fails to read the encoding.
Is this easy to reproduce?

Is this easy to reproduce?

No it happened in a 1Tb tsdb (which I eventually deleted), and it really could be a result of any failure (disk or whatever). So I don't think it's a problem things get corrupt, I think the problem is that tsdb can't work around it (or I can't tell it to work around it), I'd much prefer to lose a bit of data then have to start fresh with everything.
I'm cool with closing this if this is too broad to be a bug.

In most cases I have seen only one block is causing an issue so if you delete the entire directory you loose that date range,but Prometheus should still read the remaining.
By looking at the logs and checking the meta.json file in each folder you should be able to tell which is the offending directory.
@gouthamve suggested that we should probably add this as a cleanup command to the tsdb tool so if I find time I will try to implement this.
closing for now, but feel free to reopen if you can provide more info how to reproduce.

@krasi-georgiev In the future, how can we collect more useful debugging data to help diagnose corruption issues? Unfortunately our tsdb contains private information and we are not able to share it, unless there's some way we can anonymize labels.

that is a good idea , I will think what and how we can expose something useful .
btw I already opened the PR for the scan and delete corrupted data and just waiting for some input from the other maintainers before I start.
https://github.com/prometheus/tsdb/pull/320

One thing that surprised me is how the checksums failed to catch this. I think something more than a simple corruption is going on here. And if it is indeed a corruption, I would like the querier to catch it, error rather than panic.
Re-opening to track, will be closed when fixed.

I'm seeing the same issue in both 2.2.1 and 2.3.0:
prometheus, version 2.2.1 (branch: HEAD, revision: bc6058c81272a8d938c05e75607371284236aadc)
  build user:       root@149e5b3f0829
  build date:       20180314-14:15:45
  go version:       go1.10

-- Logs begin at Thu 2018-06-07 13:44:14 UTC, end at Thu 2018-06-07 16:29:07 UTC. --
level=info ts=2018-06-07T15:00:00.485343904Z caller=compact.go:393 component=tsdb msg="compact blocks" count=1 mint=1528372800000 maxt=1528380000000
level=info ts=2018-06-07T15:00:01.027407263Z caller=head.go:348 component=tsdb msg="head GC completed" duration=16.657809ms
level=info ts=2018-06-07T15:00:01.082731112Z caller=head.go:357 component=tsdb msg="WAL truncation completed" duration=55.186079ms
level=info ts=2018-06-07T15:00:01.141441745Z caller=compact.go:393 component=tsdb msg="compact blocks" count=3 mint=1528351200000 maxt=1528372800000
level=info ts=2018-06-07T15:00:01.926333494Z caller=compact.go:393 component=tsdb msg="compact blocks" count=3 mint=1528308000000 maxt=1528372800000
level=info ts=2018-06-07T15:00:03.57978523Z caller=compact.go:393 component=tsdb msg="compact blocks" count=3 mint=1528178400000 maxt=1528372800000
panic: runtime error: index out of range
goroutine 277 [running]:
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/chunks.(*Reader).Chunk(0xc423c76080, 0x2113006, 0x1c56c80, 0xc422088600, 0x0, 0x0)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/chunks/chunks.go:365 +0x3f8
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*compactionSeriesSet).Next(0xc424398000, 0xc421e2fb70)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:691 +0x223
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*compactionMerger).Next(0xc4202d7a40, 0xc426c2a000)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:785 +0x46b
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*compactionMerger).Next(0xc4202d7ab0, 0x15b5)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:773 +0x7f7
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*LeveledCompactor).populateBlock(0xc42015f540, 0xc423c76200, 0x3, 0x4, 0xc4202d77a0, 0x1c56cc0, 0xc426290000, 0x1c3e680, 0xc421d06a00, 0x0, ...)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:554 +0xb62
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*LeveledCompactor).write(0xc42015f540, 0x7ffc87dd0f27, 0xb, 0xc4202d77a0, 0xc423c76200, 0x3, 0x4, 0x0, 0x0)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:441 +0x5dd
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*LeveledCompactor).Compact(0xc42015f540, 0x7ffc87dd0f27, 0xb, 0xc423c76040, 0x3, 0x4, 0xdc9a7b03c3da6301, 0x6fe58eff5c2bd7e6, 0x0, 0x0)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:339 +0x693
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*DB).compact(0xc4200ac180, 0x1, 0x0, 0x0)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:419 +0x400
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*DB).run(0xc4200ac180)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:279 +0x2cc
created by github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.Open
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:243 +0x5ba

After updating to 2.3.0 (and running with debug log):
prometheus, version 2.3.0 (branch: HEAD, revision: 290d71791a507a5057b9a099c9d48703d86dc941)
  build user:       root@d539e167976a
  build date:       20180607-08:46:54
  go version:       go1.10.2

level=info ts=2018-06-07T16:55:59.948403059Z caller=main.go:222 msg="Starting Prometheus" version="(version=2.3.0, branch=HEAD, revision=290d71791a507a5057b9a099c9d48703d86dc941)"
level=info ts=2018-06-07T16:55:59.948573817Z caller=main.go:223 build_context="(go=go1.10.2, user=root@d539e167976a, date=20180607-08:46:54)"
level=info ts=2018-06-07T16:55:59.948643933Z caller=main.go:224 host_details="(Linux 4.13.0-37-generic #42~16.04.1-Ubuntu SMP Wed Mar 7 16:03:28 UTC 2018 x86_64 af3f50766d0d (none))"
level=info ts=2018-06-07T16:55:59.948716437Z caller=main.go:225 fd_limits="(soft=1048576, hard=1048576)"
level=info ts=2018-06-07T16:55:59.954446942Z caller=web.go:426 component=web msg="Start listening for connections" address=0.0.0.0:9090
level=info ts=2018-06-07T16:55:59.954461917Z caller=main.go:514 msg="Starting TSDB ..."
level=info ts=2018-06-07T16:55:59.956396679Z caller=web.go:478 component=web msg="router prefix" prefix=/prometheus
level=info ts=2018-06-07T16:55:59.956548849Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1510963200000 maxt=1511654400000 ulid=01BZV48VWFBHKZAMK620J772QQ
level=info ts=2018-06-07T16:55:59.95975985Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1511654400000 maxt=1516903200000 ulid=01C4QB6SE6B85VWNVNTSAHKWMN
level=info ts=2018-06-07T16:55:59.961946787Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1516903200000 maxt=1522152000000 ulid=01C9KZKY0XQMYPWH6R80JDZK7T
level=info ts=2018-06-07T16:55:59.964069945Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1522152000000 maxt=1527400800000 ulid=01CEGD8QHTF9K8F5825018E4VW
level=info ts=2018-06-07T16:55:59.964626111Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1527400800000 maxt=1527984000000 ulid=01CF1SCY67E74X3016N9JS3HZS
level=info ts=2018-06-07T16:55:59.964958328Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1527984000000 maxt=1528178400000 ulid=01CF7JSDMBXQCGYYBBP8TNMJTP
level=info ts=2018-06-07T16:55:59.96525028Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528178400000 maxt=1528243200000 ulid=01CF9GJXCH259TBHSHA8VQ2C5X
level=info ts=2018-06-07T16:55:59.965534207Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528243200000 maxt=1528308000000 ulid=01CFBECE07AQWSVF1F9GBP2ZMM
level=info ts=2018-06-07T16:55:59.966122625Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528372800000 maxt=1528380000000 ulid=01CFDC5XV51TZRNF3YWG8HF5FH
level=info ts=2018-06-07T16:55:59.966423978Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528308000000 maxt=1528372800000 ulid=01CFDC5Z869GY9JBK061DBM4B2
level=info ts=2018-06-07T16:56:04.288798178Z caller=main.go:524 msg="TSDB started"
level=info ts=2018-06-07T16:56:04.289014727Z caller=main.go:603 msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml
level=info ts=2018-06-07T16:56:04.296028834Z caller=main.go:500 msg="Server is ready to receive web requests."
level=debug ts=2018-06-07T16:56:10.807616966Z caller=scrape.go:703 component="scrape manager" scrape_pool=network target=http://172.17.0.1:9130/metrics msg="Scrape failed" err="server returned HTTP status 500 Internal Server Error"
panic: runtime error: index out of range

goroutine 45 [running]:
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/chunks.(*Reader).Chunk(0xc4262acf00, 0x2113006, 0x1df2e80, 0xc424da4960, 0x0, 0x0)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/chunks/chunks.go:365 +0x3f8
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*compactionSeriesSet).Next(0xc421304be0, 0xc42051a870)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:710 +0x223
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*compactionMerger).Next(0xc42030e000, 0xc423efa000)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:799 +0x46b
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*compactionMerger).Next(0xc42030e070, 0x15b5)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:787 +0x7f7
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*LeveledCompactor).populateBlock(0xc420509c70, 0xc4262ad0c0, 0x3, 0x4, 0xc42033fea0, 0x1df2ec0, 0xc42563f7c0, 0x1dd9a40, 0xc42017b180, 0x0, ...)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:573 +0xb62
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*LeveledCompactor).write(0xc420509c70, 0x7ffe26298f0a, 0xb, 0xc42033fea0, 0xc4262ad0c0, 0x3, 0x4, 0x0, 0x0)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:459 +0x3a9
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*LeveledCompactor).Compact(0xc420509c70, 0x7ffe26298f0a, 0xb, 0xc4262acec0, 0x3, 0x4, 0xfd803b242edb6301, 0x540af2e51e8907e8, 0x0, 0x0)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:345 +0x7e9
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*DB).compact(0xc4202ac3c0, 0x0, 0x0, 0x0)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:415 +0x400
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*DB).run(0xc4202ac3c0)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:275 +0x2cc
created by github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.Open
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:239 +0x593

@bonan thanks for the report.
What are the steps to reproduce this?

@krasi-georgiev see https://github.com/prometheus/tsdb/pull/344. As I said in the PR, I'm not sure how the issue can be triggered except if the chunk is corrupted?

@simonpasquier is this fixed with prometheus/tsdb#344?

One thing that surprised me is how the checksums failed to catch this. I think something more than a simple corruption is going on here. And if it is indeed a corruption, I would like the querier to catch it, error rather than panic.

@gouthamve can you point out where do you think this should be cached?

I don't think there's an issue. This is indeed possibly some corruption. But if the corruption causes the file to be too short, we don't even get to a stage where we could validate a checksum. This was simply a bug where we didn't handle an error condition properly.

yes this was my understanding as well , just didn't know if goutham had anything else in mind.
I will update tsdb and will close this when merged.

lets consider this fixed and it will be added to the next release.
Please reopen if you find that this doesn't fix the bug.
Thanks for the report again!

authors: bonan fabxc gouthamve karlhungus krasi-georgiev simonpasquier sr


text:
Expose total number of label pairs in head

Proposal
Use case. Why is this important?
For scale analysis it would be great if we either exposed a metric that reflects the total number of labels (name + value combinations) in the head block. Potentially this could also be an entry in the head block cardinality section on the status page. I don't care how I just want to get the numbers out somehow and can't today.
Side note, yes top 10 cardinality labels are useful, but may be hiding a long tail of labels that still affect the head's index, but we can't be sure until we are able to validate with numbers.
Overall context: I'm researching some other indexing strategies and it's crucial to have some real world numbers for that.
@brian-brazil @roidelapluie @bwplotka @csmarchbanks @simonpasquier

Yes!
I would go even further and have a metric of avg label value and name length. Or total. (:

Let's keep whatever roughly in sync with what the tsdbutil analyze command does.

Agreed.

I would go even further and have a metric of avg label value and name length. Or total. (:

Seems like a good case for a histogram, I would definitely be interested to see the distribution in size.

I can take this up. Is this a good first issue? Please help me with relevant packages/files that I can take a look at.
I have some context of what is included in tsdb block, please refer me to relevant files and I will revert back with my understanding.

It should be a reasonable first issue. The code is all in https://github.com/prometheus/prometheus/blob/master/tsdb/head.go
Offhand I'd say the code would go in getOrCreateWithID and gc.

no updates here for a while, let me try to help.

Hi, I'm new contributor to Prometheus. Even though there're 2 PRs already, they're all stale. That's why I'd like to take on this issue.

Hi, I'm new contributor to Prometheus. Even though there're 2 PRs already, they're all stale. That's why I'd like to take on this issue.

Please go ahead.

Please go ahead.

With #8343 it will be now available in TSDB stats page

With #8343 it will be now available in TSDB stats page

Awesome! Closing since #8343 is merged.

Awesome! Closing since #8343 is merged.

authors: brancz brian-brazil bwplotka codesome csmarchbanks khyatisoneji longngn nevill


text:
Add custom buckets to native histogram chunks encoding

https://github.com/prometheus/prometheus/issues/13487

Go test failure on windows seems transient (previously reported flaky test)

@krajorama do you plan to review this?

@krajorama do you plan to review this?

Yes of course. We've already discussed it with @zenador

I added a fix commit fix bug with wrong custom bounds when using mixed bound definitions between different series with same metric name, explanation here.

authors: beorn7 krajorama zenador


text:
Add support for utf8 names on /v1/label/:name/values endpoint

Previously, the api was evaluating this regex to determine if the label name was valid or not:
https://github.com/prometheus/common/blob/14bac55a992f7b83ab9d147a041e274606bdb607/model/labels.go#L94
However, I believe that the IsValid() function is what ought to be used in the brave new utf8 era.
Before
$ curl localhost:9090/api/v1/label/host.name/values
{"status":"error","errorType":"bad_data","error":"invalid label name: \"host.name\""}

After
$ curl localhost:9090/api/v1/label/host.name/values
{"status":"success","data":["localhost"]}

It's very likely that I'm missing something here or you were already planning to do this at some point but I just encountered this issue and figured I'd give it a go.

@newtonne
What happens if the label name contains a "/" or \n? Does the endpoint understand an escaping or quoting scheme?  In pushgateway we used the U__ scheme: https://github.com/prometheus/pushgateway/pull/689

Yeah, depending on the details of the used web framework, slashes in those RESTful URLs are definitely annoying. Some URL escaping might or might not work. (In the PGW, we decided it is too hard and used other escaping schemes. Maybe the U__ escaping is the easiest way out here.)

for reference, here are the Go implementations for escaping: https://github.com/prometheus/common/blob/main/model/metric.go#L315-L339
and unescaping:
https://github.com/prometheus/common/blob/main/model/metric.go#L367-L420

see https://github.com/prometheus/prometheus/pull/15399

Sounds good @ywwg , thanks. I was hoping to find some time over the coming days to take a look but I appreciate you taking it over.

no worries -- sorry again for losing track

authors: beorn7 newtonne ywwg


text:
Bump prometheus/client_golang to v0.9.3

It should help to get https://github.com/prometheus/client_golang/pull/568 in the next release.

Closing as @beorn7 will pack it with the tsdb update.


text:
chore(webui): optionally record anonymous telemetry

Add TelemetryTracker component to App.tsx for page view tracking
Create useTelemetry hook with VITE_PROMPTFOO_DISABLE_TELEMETRY check

I'll instrument events in the redteam setup if this looks good in the next PR

I think it's ok. Double envars is worse IMO
On Sat, Oct 19, 2024, 8:00 PM Michael @.***> wrote:

@.**** commented on this pull request.
In src/app/src/hooks/useTelemetry.tsx
https://github.com/promptfoo/promptfoo/pull/1940#discussion_r1807584700:

@@ -0,0 +1,31 @@
+import { useCallback } from 'react';
+import { callApi } from @./utils/api';
+import type { EventProperties, TelemetryEventTypes } from @./telemetry';

+export function useTelemetry() {

const recordEvent = useCallback(
async (eventName: TelemetryEventTypes, properties: EventProperties) => {

 if (!['0', 'false'].includes(import.meta.env.VITE_PROMPTFOO_DISABLE_TELEMETRY)) {

Good question. Not necessary since it's disabled in the server. This
disables the xhr from appearing in the network tab. I thought that people
might be confused / unhappy if they set PROMPTFOO_DISABLE_TELEMETRY but
still see a post to /api/telemetry with the response { success: true }
—
Reply to this email directly, view it on GitHub
https://github.com/promptfoo/promptfoo/pull/1940#discussion_r1807584700,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACLYJVT75IGPSKGDPR7BH3Z4MMDZAVCNFSM6AAAAABQIBUAMSVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDGNZZHE2TGOJUGA
.
You are receiving this because your review was requested.Message ID:
@.***>

authors: mldangelo typpo

repo: promptfoo/promptfoo

license: MIT


text:
Template output dir

if --output-dir provided, write the executed files in the directory instead of stdout

the user provides an output directory which is like a base directory, and we verify if that directory exist or panic otherwise
Now within the chart, the template files may be in different directories, we create that directory structure based on the files.

This was merged into upstream kubernetes/helm so not needed anymore here

repo: proofpoint/helm


text:
Provider can unlock staked tokens, deactivating service

As a Provider
I can deactivate my service
So that I can unlock my staked tokens

Done but needs tested.

https://github.com/prosopo-io/provider/issues/18

repo: prosopo-io/provider


text:
🛑 nftschool.dev is down

In c2fb191, nftschool.dev (https://nftschool.dev) was down:

HTTP code: 0
Response time: 0 ms

Resolved: nftschool.dev is back up in 6f6c98f after 36 minutes.

repo: protocol/upptime-pln


text:
Missing example of test spec

I am looking for an example of effective spec testing of the bots with Tourmaline.
I am willing to contribute and add to all the example bots, but I would like to have some guidance/advice before starting this endeavor.
Cheers

The main reason for a lack of tests is that testing API adapters like this is a pain. There are libraries such as webmock (for stubbing requests) and vcr (for capturing and replaying real world data), but I just haven't put in the time to set them up yet. I have been meaning to though, so I'll leave this issue here as a reminder of my failure until I get my butt into gear.

authors: razorinc watzon

repo: protoncr/tourmaline


text:
Update README.md - Link to PACTA repo has changed

PACTA now lives in a new repo (https://github.com/RMI-PACTA/pactaverse) that covers all relevant steps of data preparation and analysis. The previously linked repository will be deprecated, see https://github.com/RMI-PACTA/PACTA_analysis/pull/644
Insert URLs to the project(s) here:
PACTA
Explain what this list is about and why it should be included here:
Update an already existing project in this list to point to the up-to-date repository

Thank you for correcting this. It is often incredibly difficult to see which projects are simply inactive or which have been replaced by others.

Happy to be of help here. Will add to it if anything changes on the PACTA side of things

repo: protontypes/open-sustainable-technology


text:
Make a separate endpoint for updating schemas

Currently there's just one endpoint for both creating and updating a schema. Schema type param shouldn't be available when updating an existing schema.

#2152

Can I work on this please ?

@shubhwip sure!

@Haarolean Having looked at #2152 I think we can solve this problem just by disabling the schema type during edit. This way we can stop users to change schema type during schema edit.
Also I checked if there is any PUT endpoint on schema registry service but seems like there is nothing.
https://docs.confluent.io/platform/current/schema-registry/develop/api.html#subjects
Submitted the PR for the above change #2495

wontfix: https://github.com/provectus/kafka-ui/pull/2495#issuecomment-1231478236

authors: Haarolean shubhwip

repo: provectus/kafka-ui


text:
Fix waiting for block timeout

Broadcast in SYNC mode and poll for status (should fix transaction timeout error)
Handle account sequence mismatch error w/ retries

Codecov Report

Merging #16 (c4ab483) into main (83e63cc) will decrease coverage by 0.33%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##              main     #16      +/-   ##
==========================================
- Coverage     7.63%   7.29%   -0.34%
  Complexity       6       6
==========================================
  Files            9       9
  Lines          655     685      +30
  Branches        47      56       +9
==========================================
  Hits            50      50
- Misses         603     633      +30
  Partials         2       2

Impacted Files
Coverage Δ

...otlin/io/provenance/p8e/plugin/ProvenanceClient.kt
0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 83e63cc...c4ab483. Read the comment docs.

authors: codecov-commenter piercetrey-figure