Skip to content

GitLab

Open
Created by Johannes Reetz@johannes.reetzOwner

accounting numbers are incorrect with compound resources

Created by: cookie33

Hi,

We have a compound resource with files in both cache and archive resources. We want them counted only once per compound resource. The accounting script gives:

2017-12-20 07:00:16,217 - StorageAccounting                         - INFO - Storage space for collection: /vzSARA1/home/rods#replixRZG: 42
2017-12-20 07:00:16,390 - StorageAccounting                         - INFO - number of objects for collection /vzSARA1/home/rods#replixRZG: 14

The ils -r gives:

ils -r  /vzSARA1/home/rods#replixRZG
/vzSARA1/home/rods#replixRZG:
  1.txt
  2.txt
  3.txt
  4.txt
  5.txt
  6.txt
  7.txt
  8.txt
  t5.txt
  test00.txt
  test01.txt
  C- /vzSARA1/home/rods#replixRZG/1
/vzSARA1/home/rods#replixRZG/1:
  C- /vzSARA1/home/rods#replixRZG/1/11
/vzSARA1/home/rods#replixRZG/1/11:
  C- /vzSARA1/home/rods#replixRZG/1/11/111
/vzSARA1/home/rods#replixRZG/1/11/111:
  1.txt

So it are 12 files and not 14.

The ils -lr gives:

ils -rl  /vzSARA1/home/rods#replixRZG
/vzSARA1/home/rods#replixRZG:
  rods              1 eudat;eudatPnfs            2 2015-10-20.16:02 & 1.txt
  rods              1 eudat;eudatPnfs            3 2015-10-20.16:07 & 2.txt
  rods              1 eudat;eudatPnfs            3 2015-10-20.16:57 & 3.txt
  rods              1 eudat;eudatPnfs            3 2015-10-21.14:35 & 4.txt
  rods              0 eudat;eudatCache            3 2015-10-26.15:01 & 5.txt
  rods              1 eudat;eudatPnfs            3 2015-10-26.15:01 & 5.txt
  rods              1 eudat;eudatPnfs            3 2015-10-21.14:33 & 6.txt
  rods              1 eudat;eudatPnfs            5 2015-10-21.14:32 & 7.txt
  rods              1 eudat;eudatPnfs            3 2015-10-21.14:22 & 8.txt
  rods              0 demoResc            4 2015-12-04.17:45 & t5.txt
  rods              0 eudat;eudatCache            3 2015-10-20.12:33 & test00.txt
  rods              1 eudat;eudatPnfs            3 2015-10-20.16:01 & test01.txt
  C- /vzSARA1/home/rods#replixRZG/1  
/vzSARA1/home/rods#replixRZG/1:
  C- /vzSARA1/home/rods#replixRZG/1/11  
/vzSARA1/home/rods#replixRZG/1/11:
  C- /vzSARA1/home/rods#replixRZG/1/11/111  
/vzSARA1/home/rods#replixRZG/1/11/111:
  rods              0 eudat;eudatCache            2 2015-10-26.20:29 & 1.txt
  rods              1 eudat;eudatPnfs            2 2015-10-26.20:29 & 1.txt

There are 2 files with a copy in cache and in the archive resource, both part of the eudat compound resource. So they are counted twice. For bytes and file number.

We do not want this.

And because of this our numbers for some accounting metrics are off by more than 20TiB

Please fix this.